Markov decision processes with recursive risk measures

نویسندگان

چکیده

In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces unbounded cost under both finite infinite planning horizons. Our optimality criterion is based on the recursive application of static risk measures. This motivated by utilities in economic literature, has been studied before for entropic measure extended here to an axiomatic characterization suitable We derive a Bellman equation prove existence Markovian optimal policies. For horizon, model shown be contractive policy stationary. Moreover, establish connection distributionally robust MDPs, which provides global interpretation recursively defined objective function. Monotone models are particular.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Verification of Markov Decision Processes with Risk-Sensitive Measures

We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has been previously adopted in psychology and economics. The nonlinear transformation of the probabilities and utility functions yields a nonlinear programming p...

متن کامل

Reachability in Recursive Markov Decision Processes

We consider a class of infinite-state Markov decision processes generated by stateless pushdown automata. This class corresponds to 112 -player games over graphs generated by BPA systems or (equivalently) 1-exit recursive state machines. An extended reachability objective is specified by two sets S and T of safe and terminal stack configurations, where the membership to S and T depends just on ...

متن کامل

Iterated risk measures for risk-sensitive Markov decision processes with discounted cost

We demonstrate a limitation of discounted expected utility, a standard approach for representing the preference to risk when future cost is discounted. Specifically, we provide an example of the preference of a decision maker that appears to be rational but cannot be represented with any discounted expected utility. A straightforward modification to discounted expected utility leads to inconsis...

متن کامل

More Risk-Sensitive Markov Decision Processes

We investigate the problem of minimizing a certainty equivalent of the total or discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). The certainty equivalent is defined by U−1(EU(Y )) where U is an increasing function. In contrast to a risk-neutral decision maker this optimization criterion takes the variability of the cost into account. I...

متن کامل

Risk-Sensitive Control of Markov Decision Processes

This paper introduces an algorithm to determine near-optimal control laws for Markov Decision Processes with a risk-sensitive criterion. Both the fully observed and the partially observed settings are considered, for nite and innnite horizon formulations. Dynamic programming equations are introduced which characterize the value function for the partially observed, innnite horizon , discounted c...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: European Journal of Operational Research

سال: 2022

ISSN: ['1872-6860', '0377-2217']

DOI: https://doi.org/10.1016/j.ejor.2021.04.030